Path integral policy improvement with differential dynamic programming

Path Integral Policy Improvement with Covariance Matrix Adaptation (PI2-CMA) is a step-based model free reinforcement learning approach that combines statistical estimation techniques with fundamental results from Stochastic Optimal Control. Basically, a policy distribution is improved iteratively using reward weighted averaging of the corresponding rollouts. It was assumed that PI2-CMA somehow exploited gradient information that was contained by the reward weighted statistics. To our knowledge we are the first to expose the principle of this gradient extraction rigorously. Our findings reveal that PI2-CMA essentially obtains gradient information similar to the forward and backward passes in the Differential Dynamic Programming (DDP) method. It is then straightforward to extend the analogy with DDP by introducing a feedback term in the policy update. This suggests a novel algorithm which we coin Path Integral Policy Improvement with Differential Dynamic Programming (PI2-DDP). The resulting algorithm is similar to the previously proposed Sampled Differential Dynamic Programming (SaDDP) but we derive the method independently as a generalization of the framework of PI2-CMA. Our derivations suggest to implement some small variations to SaDDP so to increase performance. We validated our claims on a robot trajectory learning task

Effect of transmitter position on the torque generation of a magnetic resonance based motoring system

Strongly coupled magnetic resonance is most often used to transfer electrical power from a transmitter to a resonant receiver coil to supply devices over an air gap. In this work, the induced current in two receiver coils (stator and rotor) is used to generate torque on the rotor coil. The effect of the transmitter position relative to the stator and rotor receiver coils on the torque generation is studied in detail, both in simulation and experimentally. Results show a 36% to 37% gain in peak torque when properly varying the stator orientation for a given transmitter distance

Computational analysis of the effect of superparamagnetic nanoparticle properties on bioheat transfer in magnetic nanoparticle hyperthermia

P

Low-parametric Induced Current-Magnetic Resonance Electrical Impedance Tomography for quantitative conductivity estimation of brain tissues using a priori information: a simulation study

Accurate estimation of the human head conductivity is important for the diagnosis and therapy of brain diseases. Induced Current - Magnetic Resonance Electrical Impedance Tomography (IC-MREIT) is a recently developed non-invasive technique for conductivity estimation. This paper presents a formulation where a low number of material parameters need to be estimated, starting from MR eddy-current field maps. We use a parameterized frequency dependent 4-Cole-Cole material model, an efficient independent impedance method for eddy-current calculations and a priori information through the use of voxel models. The proposed procedure circumvents the ill-posedness of traditional IC-MREIT and computational efficiency is obtained by using an efficient forward eddy-current solver

Transfer learning in ECG classification from human to horse using a novel parallel neural network architecture

Automatic or semi-automatic analysis of the equine electrocardiogram (eECG) is currently not possible because human or small animal ECG analysis software is unreliable due to a different ECG morphology in horses resulting from a different cardiac innervation. Both filtering, beat detection to classification for eECGs are currently poorly or not described in the literature. There are also no public databases available for eECGs as is the case for human ECGs. In this paper we propose the use of wavelet transforms for both filtering and QRS detection in eECGs. In addition, we propose a novel robust deep neural network using a parallel convolutional neural network architecture for ECG beat classification. The network was trained and tested using both the MIT-BIH arrhythmia and an own made eECG dataset with 26.440 beats on 4 classes: normal, premature ventricular contraction, premature atrial contraction and noise. The network was optimized using a genetic algorithm and an accuracy of 97.7% and 92.6% was achieved for the MIT-BIH and eECG database respectively. Afterwards, transfer learning from the MIT-BIH dataset to the eECG database was applied after which the average accuracy, recall, positive predictive value and F1 score of the network increased with an accuracy of 97.1%

Modeling transcranial magnetic stimulation from the induced electric fields to the membrane potentials along tractography-based white matter fiber tracts

Objective. Transcranial magnetic stimulation (TMS) is a promising non-invasive tool for modulating the brain activity. Despite the widespread therapeutic and diagnostic use of TMS in neurology and psychiatry, its observed response remains hard to predict, limiting its further development and applications. Although the stimulation intensity is always maximum at the cortical surface near the coil, experiments reveal that TMS can affect deeper brain regions as well. Approach. The explanation of this spread might be found in the white matter fiber tracts, connecting cortical and subcortical structures. When applying an electric field on neurons, their membrane potential is altered. If this change is significant, more likely near the TMS coil, action potentials might be initiated and propagated along the fiber tracts towards deeper regions. In order to understand and apply TMS more effectively, it is important to capture and account for this interaction as accurately as possible. Therefore, we compute, next to the induced electric fields in the brain, the spatial distribution of the membrane potentials along the fiber tracts and its temporal dynamics. Main results. This paper introduces a computational TMS model in which electromagnetism and neurophysiology are combined. Realistic geometry and tissue anisotropy are included using magnetic resonance imaging and targeted white matter fiber tracts are traced using tractography based on diffusion tensor imaging. The position and orientation of the coil can directly be retrieved from the neuronavigation system. Incorporating these features warrants both patient- and case-specific results. Significance. The presented model gives insight in the activity propagation through the brain and can therefore explain the observed clinical responses to TMS and their inter- and/or intra-subject variability. We aspire to advance towards an accurate, flexible and personalized TMS model that helps to understand stimulation in the connected brain and to target more focused and deeper brain regions